Variable selection with ABC Bayesian forests

Few problems in statistics are as perplexing variable selection the presence of very many redundant covariates. The problem is most familiar parametric environments such linear model or additive variants thereof. In this work, we abandon framework, which can be quite detrimental when covariates impact outcome a non-linear way, and turn to tree-based methods for selection. Such screening traditionally done by pruning down large trees ranking variables based on some importance measure. Despite heavily used practice, these ad-hoc rules not yet well understood from theoretical point view. devise Bayesian probabilistic method show that it consistent regression surface smooth mix $p>n$ These results first consistency forest priors. Probabilistic assessment made feasible spike-and-slab wrapper around sum-of-trees Sampling posterior distributions over inherently difficult. As an alternative MCMC, propose ABC Forests, new sampling data-splitting achieves higher acceptance rate. We robust successful at finding with high marginal inclusion probabilities. Our algorithm provides avenue towards approximating median probability non-parametric setups where likelihood intractable.